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Abstract 

In a graphical game agents play with their neighbors on a graph to achieve an appropriate 
state of equilibrium. Here relevant problems are characterizing the equilibrium set and discovering 
efficient algorithms to find such an equilibrium (solution). We consider a representation of games 
that extends over graphical games to deal conveniently with both local a global interactions and 
use the cavity method of statistical physics to study the geometrical structure of the equilibria 
space. The method also provides a distributive and local algorithm to find an equilibrium. For 
simplicity we consider only pure Nash equilibria but the methods can as well be extended to deal 
with (approximated) mixed Nash equilirbia. 
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I. INTRODUCTION 



In the last decade we observed a rapid merging of research interests in social sciences, 
economics and computer science driven in part by the common need of analyzing strategic 
interactions in multi-agent systems. There is a growing body of empirical work evidencing 
the influence of social interactions on economic outcomes which has encouraged economists 
to take explicit account of the direct (non-market) social influences on individual decision 
making 

A general framework to study strategic interactions in multi-agent systems is game 
theory 4| . It serves to analyze situations where self-interested agents, possibly with con- 
flicting interests, struggle to get the best conditioned on (and conditioning) the behavior of 
others; the scope of the theory is to predict the strategic behavior that comes out in such 
situations, something that is usually done in the form of equilibrium concepts. Chief among 
them is the concept of Nash equilibrium (NE) that is a strategy profile in which no agent 
has incentives to deviate unilaterally . Nash equilibria can be pure or mixed, depending on 
whether the agents are required to play deterministically or are allowed to randomize among 
their available strategies. While conceptually very compelling, the concept of pure NE has 
the drawback that it does not always exist, in contrast with the universality of the concept 



BE]. 



of mixed NE whose existence is guaranteed for any finite game 

An important concern for computer scientists was to understand if Nash equilibria are 
actually efficiently computable [3]. Much work has been devoted at analyzing the computa- 
tional complexity of finding a Nash equilibrium 

This has been proved to be complete 
in a class called PPAD [l2] which contains problems believed to be hard problems [13I. Il^j ]. 
Indeed finding a Nash equilibrium typically becomes NP-hard as soon as we require it to 



satisfy certain natural properties (e.g. optimizing social welfare) 15|, |l6| or when we restrict 



it to pure strategies [13, Il8 |. 



Statistical physics has also contributed important insights and techniques to these fields 
19( 1 . For instance, in computer science it has provided powerful heuristic algorithms and 
a better understanding of the onset of computational complexity. From a physical point 
of view, a game is regarded as a system of interacting agents where an appropriate energy 
function maps the Nash equilibria to the ground states of the system. Our understanding 
of these systems has considerably improved in recent years mainly due to the concepts and 



tools developed in the study of complex systems displaying glassy behaviors [20H23]. 

From an economics modeling perspective, an interesting aspect of graphical games is to 
study the interplay between local and global interactions 
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26| . Motivated by this, we 



will study a global graphical game where agents, besides the local payoffs, receive some 
global payoffs depending on an aggregate quantity, here the average strategy of the game 
or global magnetization. We design a message passing algorithm with a polynomial time 
complexity which is exact when the graph of local interactions is a tree and show how to 
obtain an approximated algorithm to deal with the global interaction in a more efficient 
way. This algorithm resembles the two step strategy used by Horst and Scheinkman 24j to 
prove the existence of equilibria in a class of multi-agent systems with local (pairwise) and 
global interactions. Namely, we consider the magnetization as a fixed parameter, rendering 
the game an effective local graphical game, and require consistency in that we force the 
average magnetization to be equal to the fixed parameter. We test all these algorithms 
in some ensembles of random graphical games with global interactions, focusing mainly on 
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. Here we 



an extension of the best shot game or maximal independent set problem 
present upper bounds for the entropy and for the probability of having solutions which along 
with the results obtained by the cavity method of statistical physics help us to characterize 
the solution space of the problem. 

In this paper we take the constraint satisfaction approach and for the sake of simplicity 
we focus on pure Nash equilibria, but the methods can properly be generalized to deal 
with mixed equilibria. We show numerically how the information contained in the Belief 



Propagation (BP) messages 29( can be exploited to turn the algorithm into a one phase, 
fully distributed (and typically efficient) solver which directly converges to a single Nash 
equilibrium. This class of message passing solvers are called reinforced belief propagation 



(rBP)[30l. l3lj. Here we will consider the ensemble of independent payoffs that could be fully 
random or with some hidden solutions; in both cases we find typical Nash equilibria, if they 
exist, in a replica symmetric phase and easy to discover with the rBP algorithm. However, 
finding an optimal Nash equilibrium maximizing the total payoff would be still difficult as 
we enter into a replica symmetry broken phase with a more complex organization of the 
optimal solutions. 

The paper is organized as follows. In section |TT] we give some definitions that will be used 
in the paper. Section [Till includes our results on graphical games with local random payoffs 



which could have hidden solutions. In section IIVI we study graphical games with a global 
interaction that depends on the average strategy of the agents. The conclusion is given in 
section [Vj In appendix [A] we present more details on some rigorous statements mentioned 
in the text. 



II. RELATED WORKS AND DEFINITIONS 

Traditionally, a game is defined by assigning a number (i.e. a payoff) to each player 
for each possible configuration of the other players. The number of parameters involved in 
this representation grows exponentially with the number of agents in the game. Therefore, 
it is crucial to exploit additional structures that may be present in certain situations to 
find an efficient representation of the problem. Graphical games model the very common 
situation where each agent interacts only with a small subset of the whole population (see 
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- |35| for related works). The interaction structure is encoded in a graph where each vertex 
corresponds to an agent and a link between two agents indicates that the two player's payoffs 
depend on each other's choice of strategy. 

In Ref. 34| the authors provide a dynamic algorithm to find Nash equilibria on trees, 



which was later extended to deal with loopy graphs 36[. They also drew analogies with the 



belief propagation algorithm 29[ which is equivalent to the replica symmetric approximation 
in the cavity method. In fact the first phase of their algorithm can be understood as an 
instance of Warning Propagation (WP) where the probabilistic information contained in a 
belief is projected onto a boolean variable. Such a simplification allows to keep things in 
the realm of integer arithmetic and, in the case of graphical games, to prove the algorithm 
convergence. Connections to constraint satisfaction problems and Markov random fields can 



be found in 
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Consider iV players indexed by i = 1, . . . , N playing a game with given payoffs Mi(cri\agi) . 
A player payoff or utility depends on her strategy Oi G Aj and the set of strategies played 
by her neighbors in a dependency graph Q, i.e. crgj. We use di to denote the neighbor set of 
player i. In the following we consider binary strategies, that is A, = { — 1, +1}, and positive 
payoffs Mi(ai\a di ) G [0,1]. 

A strategy profile or configuration a* is a pure Nash equilibrium if for each player a* is 
the best response to the neighbors actions, i.e. 



M ii. a i\ a li) = maX M i{ a iW*di) ■ 



(1) 



Not any game possesses a pure equilibrium, but Nash theorem ensures that any finite 
game admits a mixed equilibrium |6j. Consider stochastic players where player i chooses 
strategy <7j = +1 with probability Xi G [0, 1] and the other strategy with the complement 
probability. A mixed profile x* is called a mixed Nash equilibrium if 



where the averages are taken over strategies with respect to x*. 

In this paper we shall mainly consider pure Nash equilibria. In cases where such an 
equilibrium does not exist, we ask for approximate solutions that satisfy Nash conditions 
within some tolerated errors [7|. A configuration a* is called an e-Nash equilibrium if 



where e > 0. 

We shall represent the above problem as a constraint satisfaction problem where a strategy 
profile is a Nash equilibrium if and only if it satisfies all the constraints. To this end, we 
define a constraint Ii((Ti\crdi) for each player i to check if strategy <7j is a best response or 
not. That is ij(<7j|(7gj) = 1 if a { is the best response to the neighborhood configuration agi, 
otherwise it is zero. Statistical properties of the solution space can be obtained by studying 
the following partition function 



where (3 is a parameter to optimize over the solution space. Two special limits /3 — and 
(3 — > oo give the typical and optimal welfare solutions, respectively. 

The reader can find more about physical approaches to games in 0, 12] and references 




(2) 




(3) 




(4) 



therein. 
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III. LOCAL GRAPHICAL GAMES 



We first study local graphical games where each player payoff depends only on its local 
neighborhood on the graph. In particular we consider random regular graphs (\di\ = K 
for all players). The aim is to characterize the equilibria space, for example the number of 
equilibria and their geometrical organization. To do this, we resort to the cavity method 
of statistical physics in the replica symmetric (RS) and 1-step replica symmetry breaking 
(1RSB) approximations flo| . 



A. RS equations 

In the replica symmetric approximation we assume that all solutions belong to a single 
cluster in the configuration space; there is a path connecting any two solutions such that 
neighboring solutions along the path are different only on a sub-linear number of players. 
As a result, correlations are short ranged and it is usually computationally easy to find a 
solution (a Nash equilibrium) to the problem. 

Assume that our dependency graph Q is a tree and consider cavity graph Gi-^j including 
all the nodes and edges connected to j through i. Define 7Ti_>j(ai; a., ) as the probability of 
having strategies <7j and aj in a solution of Gi->j when constraint Ij is ignored. Then one 



can easily write an equation for 7Tj_^-(<7j; Oj) relating it to other cavity probabilities 19| : 



[a^cr^oc ^ e mi Ii JJ 7rjfc_*(<7 fc ; <r<). (5) 

{crfe|fce9i\i} k£di\j 



These equations are called belief propagation equations and can be solved iteratively 
starting from random initial cavity probabilities or messages. Having the cavity probabilities 
we can obtain the free energy by the Bethe expression: 

i (ij)eg 

where AFi and AFjj are the free energy shifts by adding node i and link (ij), respectively. 
In a tree graph one obtains: 



e-^^^^U^.^ (7) 

ai <jgi jedi 

e -PAFij _ ^ -Ki^j{ai] n ; );-; ; .,(rr ; : a*). 

We expect the free energy to be asymptotically correct in locally tree graphs as long as 
correlations are short range, i.e. in a replica symmetric phase. 

The BP equations can also be used as an algorithm to find a Nash equilibrium. Rein- 
forcement is a way of doing this by progressively biasing the players to take the strategy 
that is suggested by the BP marginals [31]. The reinforced BP equations are 



TTi^j^CTj) oc [7r i (cr i )] r ^ e? Mi Ii ] [ n k ^i(cr k ;ai), (8) 

{a k \kedi\j} kedi\j 

{cr k \k£di} kddi 

where r > is the reinforcement parameter. One can start with random initial values 
for the messages and update them according to the rBP equations in a random sequential 
way. At the beginning we set the reinforcement parameter to zero and increase its value 
slowly while the system converges to a solution. Notice that for r — > oo any solution of the 
problem is a fixed point of the above equations. The rBP equations thus suggest a local and 
distributive message passing algorithm to approach a Nash equilibrium. 



B. 1RSB equations 

For simplicity let us take the limit f} — > where — (3F is equivalent to entropy S. In 
the one-step replica symmetry breaking framework we assume there exist an exponentially 



large number of clusters of solutions 19(. This number is given by the so called complexity 
or configurational entropy by e Nj: . Clusters can have different internal entropies (or sizes) 
s = S/N and E(s) is used to indicate the complexity of a cluster of size s. The so called 
dominant clusters are those that maximize E(s) + s; with high probability a randomly 
selected solution belongs to these kind of clusters (though any specific algorithmic strategy 
would end up in clusters which are not necessarily the dominant ones). It is useful to 



introduce Lagrange multiplier m and work with generalized free energy m$ = E(s) + ms. 
This allows to obtain the complexity by an inverse Legendre transform after computing the 
generalized free energy As long as we are in the RS phase (even if the solution space is 
clustered) the relevant clusters are those corresponding to m — 1. This complexity goes 
continuously to zero at the thermodynamic RSB phase transition, after that the physical m 
would be less than 1, its value determined by the point of zero complexity. 

We assume that each BP fixed point corresponds to a cluster or state of the system. The 
1RSB equations give the statistics of BP messages among different clusters fl9 ] 

P^fri^j) <x J J] dP^^e^^S^-BP^), (9) 



k€di\j 



where 



= I{ IT ^iOfc; *)- ( 10 ) 



&i,o'di kedi\j 

Here m is called Parisi parameter and is to control the entropy density. Given the cavity 
distributions Pi^j{^i^j) we obtain the 1RSB free energy in the Bethe approximation 

$ = E A ^- E A %> ( n ) 

i (ij)eG 

where 



/ HdP^^e™^, (12) 

J jedi 

e mA ^ = J dP .,{-; v )r//> ; .,)^" A -<. 

The 1RSB free energy is related to the entropy and complexity by m$(m) = S(s) + ms 
where 



0E(a) 2 d<f(m) 
m = -- Br , S( S ) = -m^— . (13) 

The total 1RSB entropy is given by E(s) + s computed at the physical m. As long as we 
are in the RS phase the 1RSB entropy computed at m — 1 is equal to the BP entropy. 
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FIG. 1. BP entropy in the local graphical game with random payoffs. 



For a given graphical game we can solve the 1RSB equations with the population dynamics 



technique [20|, |22j. We represent the probabilities Pi^jiji^j) on each directed edge of the 



graph with a population Pop^j of BP messages {7r a |a = 1, . . . ,N p }. To update Pop^j, 
we first select randomly messages n ak from Popk^i for k G di\j. Then the new message 



7T 



and the entropy shift A5^j are computed according to the BP equations, and with 
probability oc e mA5i ^ a randomly selected message form Pop^j is replaced with the new 
one. After converging to a stationary state we compute the average quantities by taking 
samples from the populations. 

With the same scheme, one can write the 1RSB equations for other values of /3, replacing 
the entropy with free energy. 



C. Random payoffs 

Let us start with fully random payoffs where each element Mi(ai\aQi) G [0, 1] is a uniform 
random number independent of the other payoffs. As figure [I] shows, here the BP entropy 
is positive only for approximate solutions with e > e c (N). For smaller e the BP algorithm 
finds contradictory messages, something that usually happens when the solution set is empty. 
Moreover, the critical e increases with N and finally for iV — > oo we would have e c = 1. 
Notice that for e > 1 any configuration is a solution to the problem. 

It is easy to compute the average number of solutions or annealed entropy by averaging 



over the randomness in the payoffs 



s annealed = ]_ = j^) + ^ (ll^l -j . (14) 

The above annealed entropy, displayed in figure [XJ, provides an upper bound for the 
correct number of solutions. However, this entropy is always greater than zero, except for 
exact solutions at e = 0. To understand why the approximate solutions do not survive in 
the thermodynamic limit we need to resort to another argument |41|: Consider an arbitrary 
region Q of the graph Q. Suppose that we fix the strategies of boundary players dfl to ctqq. 
Depending on the boundary state and payoffs, we may have no best response solution for 
the players in Q. And this could happen for all boundary configurations agn- We denote 
the probability of this event by P n0 soiution(^) ■ Consider a collection C of Nq disjoint regions 
in Q. Then, the probability of having a solution is overestimated by 



Psolution — J^J(1 PnosolutioniS^l)\ 
I 

The simplest choice is when each region consists of a single player. Obviously in this case 
Pnosoiution = 0, which leads to a trivial inequality for P so i u tion- We can choose a larger region 
consisting of two neighboring players in the graph. For uniform and independent random 
payoffs one finds P n osoiution(Q) < (1/8) 2 ' 9 " 1 . As long as \d£l\ is finite and Nq = O(N) this 
results to an exponentially small probability of having a solution. 

Any way, given a finite game, we find that BP always converges as long as e > e c (N); 
the system is in the RS phase with zero complexity and we can easily find an e-Nash equi- 
librium using our reinforced BP equations. In figure [2] we compare the success probability 
of this algorithm with another message passing algorithm, similar to warning propagation, 
introduced in Ref. 34j. Here, a simple heuristic algorithm like Best Response (BR) does 
not converge to a solution. In the BR algorithm, we start from an initial strategy profile 
and as long as some players are not satisfied we randomly select one and update its strategy 
to the best response. 

As mentioned above the random payoff ensemble has a trivial thermodynamic limit. To 
get around this problem we shall consider random games where nontrivial solutions exist. 
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FIG. 2. Success probability of the rBP algorithm compared with the WP algorithm of Ref. 
in the local graphical game with random payoffs. The data have been obtained by running the 
algorithms on 100 problem instances defined on random regular graphs of degree K = 3. 



D. Random payoffs with hidden solutions 



We can always modify random payoffs to ensure that our game has at least some pure 
Nash equilibria. Suppose that we want configuration a* to be a solution to the problem. 
Then we modify the payoffs in the following way: for each player i, if necessary, we swap 
the two values M^a^a^) and Mj(— cr*\a* di ) to satisfy the Nash condition for the player. 

Let us take the uniform and independent random payoffs to see how the picture changes 
when we plant random configuration cr* into the solution space. For each player we choose 
a* with equal probability from { — 1,+1}. Figure [3] displays the BP entropy computed on 
some large instances of the problem. Interestingly, planting only one solution is enough to 
have an exponential number of pure Nash equilibria at e = 0. Moreover, these solutions 
do not disappear in the large N limit, as it happens for games with random payoff. To 
understand this consider the planted solution and a pair of neighboring players. There is 
a finite probability that after nipping the two strategies we get another Nash equilibrium. 
Moreover, there is an extensive number of independent such players that could result to an 
exponential number of solutions close to the planted one. 

Introducing temperature into the problem makes the phase diagram more interesting in 
that we observe a critical line /3 c (e) separating the RS and RSB phases in the (3 — e plane, 
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FIG. 3. BP entropy and average payoff in the local graphical game with a hidden solution. 



see figure HI BP does not converge for /3 > /3 C , signaling an RSB phase transition. Moreover, 
there is a finite temperature-gap for any e, meaning that finding a typical e-Nash equilibrium 
is an easy task. Figure [5] shows how the entropy and average payoff change with /3 for exact 
Nash equilibria. For comparison we have also given the average payoff of solutions obtained 
with the rBP algorithm. In figure |3] we also display the average total payoff as a function 
of e for different values of 0. We observe a maximum appearing in the average payoff as 
we increase /3\ obviously for small (3, more accurate solutions have larger total payoff than 
solutions satisfying Nash condition at a larger e, since Nash condition is already maximizing 
the local payoffs. However, when /3 is large enough we select those strategies maximizing 
the total payoff and with a larger e we have more space to find a better global maximum. 
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FIG. 4. The phase diagram in the local graphical game with a hidden solution. The line has been 
obtained by checking the convergence of BP equations on a single instance of size N = 10 5 and 
degree K = 3. For (3 > (3 C the equations do not converge in T max = 1000 iterations. 

A more accurate estimate of the entropy and of the total payoff is obtained by considering 
replica symmetry breaking. Figure shows the m = 1 complexity computed in the 1RSB 
approximation at different temperatures. The system is in the RS phase for small 0; BP 
converges, complexity is zero and the total 1RSB entropy is equal to the BP entropy. For 
larger (3 we have replica symmetry breaking, more precisely we enter into a condensed phase 
where only a finite number of solution clusters are relevant. In this case, as we see in the 
figure, the m = 1 complexity is negative but the relevant m will be less than 1 where 
complexity is zero. 

Another way of planting solutions is to modify the random payoffs to mimic the con- 
straints in an already known problem. In this case, we go through all the neighboring configu- 
rations of a player and if necessary swap the two values Mi(<Ji\<jQi) and Mj(— <Ji\agi) according 
to the constraints. An example that we will later study in this paper is the maximal indepen- 
dent set (mIS) problem where a player plays +1 only if all its neighbors play —1. Regarding 
the payoffs, it means that for each player we need to have Mj(+l|Vj G di, crj = —1) > 
Mi(-l|Vj E di, Oj = -1) and M;(-l|3j e di, a j = +1) > Mi(+l\3j E di, a s = +1). It 
is easy to obtain a typical solution for this problem by running the Best Response algorithm. 
What is difficult is to find an optimal solution, for example with a large number of active 
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FIG. 5. BP entropy and average payoff in the local graphical game with a hidden solution. The 
vertical line separates the RS (small (3) and RSB (large (3) regions. The rBP average payoffs are 
results of averaging over at least 50 solutions obtained with the rBP algorithm. 



players. Algorithms based on the BP equations help us to find good optimal solutions in 
large problem instances [42|]. In figure [7] we show the entropy of solutions with magnetization 
m = jfi^iO'i) in the region that BP equations converge. 



IV. GLOBAL GRAPHICAL GAMES 

In a global graphical game the payoffs depend, besides the local neighborhood, on the 
global state of the system, for instance as 
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FIG. 6. m = 1 complexity in the local graphical game with a hidden solution. The data are 
obtained by solving 1RSB equations with population dynamics on a single instance of the problem. 
On each directed edge of the graph we have a population of size 1000. 

Mi = M^inWai) + M***^), (16) 

where g(a) is an aggregate quantity depending on the strategy profile. As for local games 
the total number of solutions can be written as 

Z = J2Y[l i (<T i \<T 9i ,g), (17) 

ct i 

where is an indicator function to check Nash condition for player i. An interesting 
example is 

M global = ha . m ^ m = ^ (Tj, (18) 

i 

where players will receive more than their local payoffs if they are in majority or minority, 
depending on the sign of global field h. These kind of interactions are similar to mean field 
models in statistical physics but in a different setting. It also makes sense from a social 
point of view to have such global incentives {43 1. 

The sole global problem Mj = hoim has only two solutions with magnetizations m — ±1 
for h > and C(N, N/2) solutions with magnetization m = when h < and iV is even. 
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FIG. 7. Entropy as a function of magnetization in the local graphical game with hidden solutions. 
We plot the entropy only if the BP fixed point is stable. 

M 

Here C(N,l) = is the Binomial coefficient. 

W 

One could consider other global terms more suited to the local problem. For instance, 
we might be interested on the total parity of solutions, where payoffs are given by 

Mi = M^ al (ai\a di ) + h J] en. (19) 

i 

In statistical physics these type of interactions are studied to model structural glasses. 
In this case the global problem Mj = h cr, partitions the configuration space to even and 
odd parity solutions for h > and h < 0, respectively. 
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In this paper we shall study the former problem where total activity or magnetization 
m determines the global payoff. For the sake of simplicity, in the rest of the paper we shall 
work in the replica symmetric approximation and set j3 = and e = 0. 



A. Rigorous results 

Suppose that local payoffs take integer values in {0, 1} and there is no degeneracy, that is 
M l i ocal (a i \aQ i ) 7^ M| oca '(— Gi\(Jdi)- This special case helps us to understand the problem better 
when studying real and random payoffs. Consider a strategy profile a with magnetization 
m. Flipping the strategy Oi of player i results to the following change in her payoff: 



AMi = AM l i ocal - 2h(<Tim-^), (20) 

where the 1/N term comes from the change in total magnetization. In order to simplify 
the arguments, we will consider only strict solutions, i.e. those with AM; < 0. We say a 
player is locally happy if AM- OCQi < and group the players in distinct sets H + , U + ,U^ 
for locally happy (if) and unhappy (U) players with plus and minus strategies. It is easy to 
see that players in H + satisfy the Nash condition only if: 

m -Jj<WV h<0] (21) 
m -Ji > ~WV h>0] 
The above conditions define two lines in the m — h plane. Similarly we can write the 

conditions for the other sets. From these we conclude that: 

Remark (1): For — 2 (±-i/n) < h < 2 (i+i/n) an y l° ca l solution is also a solution of the 
problem. A local solution is a solution of type (H + ,H-). The entropy of local solutions 
survived at global field h is: 



s loca \h)= r max s ioca \m), (22) 

— m* (h)<m<+m* (h) 



where 



' x _1 sgn(fe) 

s '2|/i| N 

We defined s local (m) as the entropy of local solutions (i.e. at h — 0) having magnetization 



m * (/l) = min(l '2W~ iV '• (23) 



m. 
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Remark (2): For h < —N/2 and h > N/2 only global solutions remain. For positive h 
the two global solutions with m = ±1 appear at h — 2 {i-i/n) ' wnereas f° r negative h the 
global solutions with m = appear at h — —N/2. 

Remark (3): For —N/2 < h < N/2 one may have mixed solutions. These are solutions 
of type (-£/+, U+) (for hm > 0) or (H + , H_, UJ) (for hm < 0). That is we can not have 
solutions with both sets U + and LL non-empty. There is no mixed solution for h > 0, and 
if there is a mixed solution for fa < —1/2 it must have magnetization m = ifjkj - 

When the local payoffs are uniform random numbers in [0, 1] satisfying the maximal 
independent set constraints, we have: 

Remark (4): There is no solution with magnetization m E\2p max — 1, 0[ for h > 0, where 
pmax denotes the size of the maximum independent set in graph Q. Moreover, the probability 
of having a solution is zero in the thermodynamic limit when < 2h(m ± 1/N) < 1. 

The reader can find more about these remarks in appendix [A] 

B. Annealed approximation 

Given the ensemble of local payoffs we can compute the average number of solutions in 
a graphical game as 

(Z) = (e Ns ) = J2U^- ( 24 ) 

(7 i 

The average is taken over the randomness in the independent local payoffs. The convex- 
ity of exponential function ensures that s annealed = \n(Z) is an overestimate of the average 
entropy (\nZ). In a global graphical game the constraint Ii depends also on total magne- 
tization density m, therefore, we will restrict the above equations to the subspace of fixed 
M = a %- Then the average number of solutions reads 

(Z) = J2 E e Ns ^ H ^ H -» J] p(H a ) N ^p{U a ) N ^\ (25) 

M N(H + ),N(H_) <r=+l,-l 

where e Ns i m M H +)M H -)} [ s the number of configurations with specified densities, e.g. 
n(H + ) = N(H + )/N. Notice that the other two densities are not independent but given 

by 
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N(U + ) = — N(H + ), (26) 

N — M 
N(U-) = —j—-N(H.). 

Moreover, as described in the previous section we can not have all densities nonzero; 
depending on the sign of hm one of the two quantities N(U + ), N(U_) should be zero, such 
that at the end there remains only one independent parameter N(H + ) or N(H_). For given 
h and m, the probability that a locally happy player is satisfied by the total payoff is 



p(if_) = Pr(AM' oca ' > 2h(m + l/N)), (27) 
p(H+) = Pr(AM local > -2h(m - l/N)). 



And if the player is locally unhappy 



p(U-) = Pr(AM local < -2h(m + 1/N)), 
p(U+) = Pr(AM local < 2h(m - l/N)). 



(28) 



For uniform random payoffs in [0, 1] we have 



PrflAM' 



local I 



> e 



1 e < 0; 

(1 -e) 2 < e < 1; 
e > 1. 



(29) 



The above quantities are enough to compute the average number of solutions for random 
payoffs. Let us separate the two cases of positive and negative fields. For h > : 



' C(N, (N - M)/2)p{H + )^ N+M ^ 2 [l + p(U_)]( N - M y 2 M < -1; 

C(N, (N - M)/2)p(H + ) i - N+M ^ 2 M = -1; 

(Z) = 7^ I C(N, N/2)p{H„) N / 2 p{H + ) N l 2 M = 0; 

C(N,(N + M)/2)p(H_)( N - M y 2 M = +l; 

k C(N,(N + M)/2)p(H_)( N - M y 2 [l+p(U + )]( N+M y 2 M>+1. 



(30) 
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And for h < 



(Z) 



1 

2^ 



C(N, (N + M)/2)p{H^ N - M V 2 [l + p{U + )^ N+M ^ 2 M < 0; 
C(N,N/2)[l+p(U^] N / 2 [l+p(U + )} N / 2 M = 0; (31) 

C(N, (N - M)/2)p(H + Y N+M y 2 [l + p(U^ N ~ M ^ 2 M > 0; 

Notice that given h the entropy is symmetric with respect to m. It is easy to see that 
the only positive contribution to the entropy comes from m = configurations when h < 
and scales with iV: 



annealed 



= In 



1 



N 



(32) 



In the other cases, the annealed entropy as a function of magnetization is always less 
than or equal to zero. Consider for example the case h > such that < 2hm < 1, we have 



_ ln(2 )-(l^W ^ 



1 — ra\ f 1 — m 
In 



(33) 



The above entropy is zero only at m = 0, or at m = m (h) if h is greater than critical 
value h c ~ 0.25 determined by the following equation 



mo = tanh(/i 9 — hi), (34) 

/i, = 2/i(l - m )- },~ 2km °, 2 2 + ^ ln[l - Ahm + Ah 2 m 2 l 

1 — AhrriQ + Ah z rriQ 2 

h ° = 2,1(1 + m °» l + 4 ftm „-tf m g + 2 b|1 + ikm " ' ^ 

The nontrivial magnetization itlq approaches to 1 as global field h reaches the value 1/2. 

The situation is a bit more complex when the local payoffs have some structure. The 
difficulty comes from computing s[m,n(H + ),n(H_)] which was trivial for random payoffs. 
When dependency graph Q is a chain we can compute this entropy exactly. However, for 
arbitrary graphs we will estimate it in the Bethe approximation by solving the following 
problem: 
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Z(x,y) = J2e xN ^ +yN ^I(U T = 0). 



(35) 



a 



Here r = —sgn(hm) and I(U T = 0) is an indicator function to have U + or U- empty, 
depending on the sign of hm. The entropy is obtained by a Legendre transformation after 
computing the free energy of this problem for appropriate values of fields x and y. Figure 
E] shows the annealed entropy obtained in this way for random regular graphs of degree 
K = 3 with local mIS constraints. Notice the small entropy maximums appearing close to 
the global polarized solutions when h is approaching h c ~ 0.43. We remind that according 
to remark (4) the entropy is zero in thermodynamic limit for < 2h(m ± l/N) < 1. As 
for random local games in section IIII CI here the annealed entropy does not give the correct 
behavior. However, it is still useful in that we obtain a qualitative picture of the entropy in 
finite size systems. 

C. Global algorithms 

The global graphical game can be treated like the local one by introducing a global 
constraint fixing the global quantity. Suppose that the global payoff depends on a global 
variable g G A ff . We write the following partition function to count the Nash equilibria 



where as before It checks for Nash condition and I g is an indicator function to fix quantity 
g = g(&). We can then write the standard BP equations regarding I g as another constraint in 
the problem. However, in this way we introduce a large number of loops into the problem, 
which destroys the BP exactness even when the original graph Q is a tree. To preserve 
this property here we follow another strategy in which the global constraint is broken to 
local ones in the expense of introducing new variables. Consider global quantities that their 
computation can be partitioned into smaller local computations. This is the case for example 
when g is total magnetization or parity. Then we introduce cavity variables g^j which are 
passed along a spanning tree T of graph Q. These variables are determined by other cavity 




(36) 



y 



a 
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FIG. 8. Annealed entropy as a function of magnetization in the global problem with local mIS 
constraints. The entropy has been computed for random regular graphs of degree K = 3. 

variables as g^j = g(<Ji, {gk^i\k 6 di \j, T}). For instance, when g is total magnetization, 
the cavity magnetizations are given by g^j — + Y^kediXj t 9k-H- Now the global partition 
function can be rewritten as 



z= ^2 ^Yi^g^iWdugi), (37) 

{Si-*-jl(«)6T} g_ i 

where Ii g is to check the Nash condition and outgoing cavity variables {g^j\j G di,T}. 
Notice that each player computes its estimate of the global quantity g\ = gipi-, {gk^i\k £ 
di, T}) locally after receiving the incoming cavity variables. The BP equations with new 
variables read 
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FIG. 9. Comparing the exact entropy with the annealed one for a chain of players in the global 
problem with local mIS constraints. The data are results of averaging over 10 instances of random 
payoffs. 



~, >M,-g, .r a r^:, v 2^ iig(vMdhdi) [I Kk-+i{<7k,gk->i;<7i,gi-+k)- (38) 

{(T k ,g k ->i\kedi\j} k€di\j 

The time complexity of this algorithm is NK\A g \ hmax where k max is the maximum degree 
in spanning tree T. Actually, the complexity can be reduced to NK\A g \ 2 if we pass the 
messages g^j along a spanning chain which is used just to compute the global quantity. 
Nevertheless, the algorithm is still computationally expensive specially when |A S | is large. 

In figure [9] we compare the exact BP entropy computed in this way with the annealed 
entropy in a small chain of players, when g = m and |A 9 | = N. 



D. Local algorithms for global games 

In this section we present another way of dealing with the global payoff when it depends 
on the total magnetization. Considering h and m as fixed parameters in the payoffs, we 
reduce the problem to a local one but with modified conditions for solutions. The partition 
function for this local problem reads 
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Z{m) = Y J \{ I iWi\a di ,m). (39) 

a_ i 

Clearly the entropy computed in this way is an upper bound for s(m), the entropy of 
solutions with magnetization m in the global problem. The reason is that here m is just a 
parameter which is not necessarily the total magnetization. Indeed, we can do better than 
this by introducing an external field to really fix the total magnetization to m 

Z(m) = Y,e cT ' i<Ti T[li{^m,m), (40) 
where x is chosen such that 

ld\nZ{m) 

m = T7 r, (41) 

N dx K ' 

In the Bethe approximation the average magnetization is computed from the cavity mes- 
sages satisfying BP equations 



[<Ti\aj)ac ^ e xai Ii(ai\a di ,m) | [ n k -n(a k \ai). (42) 

{a k \k£di\j} k£di\j 



In this way we obtain a better estimation of s(m) as displayed in figure HU1 This method 



of fixing magnetization has already been implemented in Refs.j44j446|. 

The above entropy can also be computed with the population dynamics technique. To do 
this we represent the set of BP messages in a graph by a large population of messages. The 
population is updated by selecting randomly K — 1 messages 7r ai and computing a new BP 
messages according to the BP equations. Then a randomly selected element of population 
is replaced with the new message. To fix the magnetization we also update field x such that 
the expected magnetization in population is equal to m. 

There are some subtle points here to mention about the population dynamics. First note 
that in the population dynamics we do not have the small payoff change 2h/N in the Nash 
conditions; we are at the thermodynamic limit iV — > oo, and h is finite. In all the numerical 
simulations we will work with a global field of order 1, which is reasonable since the local 
payoffs are real numbers in [0,1]. At the first sight the small term 2h/N seems irrelevant 
for large problem sizes but adding this correction could eliminate some solutions when h is 
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FIG. 10. BP entropy as a function of magnetization in the global problem with local mIS con- 
straints. The data have been obtained with population dynamics with N p = 2 x 10 5 . Here the 
errorbars show the difference between the fixed and converged magnetizations. 

positive. We recall that the Nash condition for player i is AM- ocal — Ihoim < —2h/N; and 
this is a stronger condition for a solution than AM| ml — Ihoim < when h > 0. Indeed, 
there is a finite probability to miss a solution by adding the small term 2h/N even in the 
large N limit. Secondly, the size of population that we use to represent the statistics of BP 
messages is finite. It means that, even if in the thermodynamic limit we do not have any 
solution, we may still observe a positive entropy due to the finite size of the population. What 
we can do is just to work with the largest population allowed in our numerical simulations. 

The local problem can also be used to converge the system to a problem solution by 
applying reinforcement. Here are the reinforced BP equations 



ni-tjfaaj) oc [7r i ((T i )] r ^ lMi\a di ,m) j [ 7r fc ^(cr fc ; cr;), (43) 

{a h \k£di\j} k£di\j 

Zi(<Ti) oc [Tr i ((T i )) r ^ lifaWdi,™) Yl 7r fe ^ (o- fc ;o-j). 

{a k \k£di} kedi 

The algorithm works by computing m = ^[^(+1) — 7Ti(— 1)] from the BP marginals at 
each iteration and using that as an estimation of total magnetization in the Nash conditions. 
Remember that m can be computed locally by passing appropriate messages g^j along a 
spanning tree. Figure [11] compares the magnetization of solutions found in this way with the 
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Best Response algorithm in a problem with local mIS constraints. We see that even for a 
large graph of iV = 10 4 players, there are still some Nash equilibria for < 2hm < 1, where 
the solution set is asymptotically empty. The Best Response algorithm finds easily a solution 
when the global field is positive but it does not converge for negative h. Using the above 
rBP algorithm we could obtain different kinds of solutions depending on the reinforcement 
parameter. The solutions lay, of course, in the region that BP equations converge. In figure 
[TTJwe also show the typical entropy for positive values of the global field. It seems that both 
the entropy and magnetization approach continuously to their limiting global values. 



V. CONCLUSION 

We used the cavity method in conjunction with rigorous bounds to study random graphi- 
cal games with local and global interactions. We analyzed the phase diagram of the problem 
and presented some local message passing algorithms which allow to find efficiently a Nash 
equilibrium. 

More specifically, we studied graphical games with local payoffs coming from the maximal 
independent set problem and global payoffs which depend on the average strategy over the 
whole graph (the so called total magnetization in the physics jargon). Introducing the 
global interaction resulted to a new set of equilibria which are a mixture of locally happy 
and unhappy players. In summary: (i) Using rigorous arguments and the annealed entropy, 
we showed that these equilibria cannot be present in some regions of the phase space in the 
thermodynamic limit, (ii) We observed an exponentially large number of these equilibria 
for positive global fields and negative magnetizations which we conjecture survive in the 
thermodynamic limit. Indeed, a simple heuristic algorithm like the Best Response is able 
to find such a typical solution in very large problem instances. The entropy and the total 
magnetization of the typical equilibria decrease continuously as we increase the global field 
starting from zero, (iii) Numerical simulations and annealed approximation results show 
the existence of a critical value of h above which a cluster of solutions dominated by the 
global interaction appears. Close to this value of the global field, the typical entropy and 
magnetization decrease very rapidly, separating the regions governed by the local and global 
interactions. 
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FIG. 11. Upper plot: average magnetization of solutions found by the rBP and BR algorithms in 
100 instances of the global problem with local mIS constraints. The vertical line shows the point 
where globally dominated solutions appear. The inset displays the behavior for different sizes. 
Lower plot: typical entropy computed with population dynamics. 
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Appendix A: More details on Remarks (1) to (4) 



On Remark (1): Consider a solution of the problem with magnetization m and global 
field h. In a local solution players are either in set H + or H_. For a plus player the Nash 
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condition is AM local —2h(m—l/N) < which for happy players means — 1— 2h(m— l/N) < 0. 
Notice that for integer payoffs in {0, 1} a happy player has AM local = — 1 whereas for an 
unhappy player AM local = +1. Therefore, a happy plus player is plying the best response 
as long as 



m -Jj<WV h< ^ ( A1 ) 

For a happy minus player the Nash condition is —1 + 2h(m + l/N) < which is satisfied 
when 



m + ^>~WV h<0] ( A2 ) 

m + j; < m> h>0 ' 

To have a local solution we just need —m*(h) < m < m*(h) where m*(h) = min(l, — 
sgn ^ ) , If \h\ < 2(i+s g n(fe)/jv) we m* = I, that is all local solutions survive after adding 
the global incentives. 

On Remark (2): According to the above arguments, to have both happy plus and minus 
players when h > N/2 we need at the same time m > and m < 0, which is impossible. 
But we can have all plus or all minus solutions, that is the two global solutions for h > 0. 
Indeed these two solutions appear at h — 2 (i-i/n) wnere unhappy players are allowed. To 
see this we note that a player in U+ is satisfied if 



The all plus solution is possible if 1 > ^ + that is h > \ + j^. For players in U- we 
have 



m + ^>wv h< ^ (A4) 

m + J!< -2^' /l>0; 

The all minus solution is possible if— 1 < — ^ — jj, that is h > 2 (i~i/n) • 
On the other hand, to have the m = global solutions for h < 0, we need > ^ — 
which implies \h\ > N/2. 
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On Remark (3): From the above equations we see that to have players in both U + 
and U- when h > 0, we need m > + jj and m < —-^n — which is not possible. 
Indeed one could have only unhappy plus players if m > or unhappy minus players if 
m < 0. But a mixed solution should contain both plus and minus players, otherwise it would 
be completely polarized global solution. Considering the case m > 0, the conditions for 
unhappy plus players and happy minus players give contradictory inequalities m > m + 
and m < ^ — In the case m < 0, the conditions for unhappy minus players and happy 
plus players give contradictory inequalities m > ~2[h\ + ~k anc ^ m < ~2[h\ ~~ Tf' Therefore, 
we can not have a mixed solution for positive global fields. 

When h < 0, to have non-empty sets U+ and U- we need m < — + and m > m — ^. 
Again we can only unhappy plus players if m < or unhappy minus players if m > 0. In 
other words, we can only have solutions of type (H + , H-, U + ) if hm > and (H + , H-, Z7_) 
if /im < 0. Considering the two cases m > and m < we find m — ^ < m < + 
and — 2pT| — N < m < — 2pT| F' res P ec tively. That is, if there is a mixed solution it should 
have magnetization ijjjkf anc ^ so ^1^1 > ^' 

Note also that for both h > and ft < we can not have an m = solution of type 
(#+, if-, U+, U-) as long as \h\ < N/2. 

On Remark (4): Consider the maximal independent set problem on graph Q and let 
Pmax be the size of a maximum independent set. Thus, all local solutions have a magne- 
tization less than 2p max — 1. Consider the global problem when h > 0. According to the 
previous remarks, if there is a solution of magnetization 2p max — 1 < m < to the problem, 
it should be of type (H + , H-, UJ). But this is not possible because we can always change the 
state of a player in £/_ to +1, increasing the magnetization and still respecting the mIS con- 
straints. In other words, such a configuration would lead to local solution of magnetization 
m > 2p max — 1, which is of course a contradiction. 

The other part of remark (4) can be proved by bounding the probability of having a 
solution, as we did in section Ull CI for random payoffs. Consider a region Q which is a chain 
of three neighboring players (j,i,k). Here we show that if < 2h(m ± 1/N) < 1, then for 
any boundary configuration <jqq there is a nonzero probability of having no solution in the 
region. 

Let us start by eliminating the two configurations (+1, +1, — 1), (— 1, +1, +1) in which 
player i is always locally unhappy, independent of the boundary configuration. To do this we 
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need \AM\ ocal {ai, agi)\ > 2h(m — l/N) that could happen with a nonzero probability as long 
as 2h(m—l/N) < 1. Now consider configuration (+1, —1, +1) where player i is always locally 
happy. Again we can avoid this solution by choosing | AM l ° cal (o-i, a di )\ < 2h(m+l/N), which 
is possible if 2h{m + 1/N) > 0. 

Then consider three configurations (—1, +1, —1), (—1, —1, +1), (—1, —1, —1) where player 
j could be locally happy or unhappy. In each case we can eliminate the solution by choosing 
\^M 1 f ca \a j ,a dj )\ < 2h(m + l/N) or | AM l ° cal (<x,- , o~Qj ) | > -2h(m + l/N). This can be done 
with a nonzero probability if 2h(m + 1/N) > 0. 

There remain two configurations (+1,-1,-1) and (+1,+1,+1) where player k could 
be locally happy or unhappy. Again we can choose \AM l k oca \a k , adk)\ < 2h(m + 1/N) or 
\AM l k ocal (a k ,a dk )\ > -2h(m + 1/N) to eliminate (+1,-1,-1) if 2h(m + 1/N) > 0. And 
we choose \AM l k ocal (a k , a dk )\ > 2h(m-l/N) to eliminate (+1,+1,+1) if 2h(m + 1/N) < 1. 
Therefore, we have a nonzero probability Pnosoiution{Q) which in the thermodynamics limit 
gives an exponentially small probability of having solution P so i ution < (1 - P n osoiution(ty) Nn ■ 
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